In human-computer interaction, one of the most basic elements involves selecting and activating a target object using a pointing device. Target object activation and selection are involved in opening a file with a mouse “click”, following a world wide web link, choosing a menu item, redefining a typing or drawing insertion position, and other such operations. Selection and activation are two stages of interaction. Selection refers to the stage where an object is chosen, and activation refers to invoking the function of the object. Engineers and scientists have developed many different approaches to target object interaction. One of the most popular target object interaction devices is the computer mouse. Although computer mice are practically essential with today's computers, intense use can be fatiguing and time consuming. Further, the use of a mouse is difficult for handicapped users or users with limited hand mobility due to, for example, carpal tunnel syndrome or arthritis.
Despite these limitations, further improvement of mouse-based target object selection and activation systems has been difficult. One interesting idea for possible improvement uses eye gaze tracking instead of mouse input. There are several known techniques for monitoring eye gaze. One approach senses the electrical impulses of eye muscles to determine eye gaze. Another approach magnetically senses the position of special user-worn contact lenses having tiny magnetic coils. Still another technique, called “corneal reflection”, calculates eye gaze by projecting an invisible beam of light toward the eye, and monitoring the angular difference between pupil position and reflection of the light beam.
With these types of gaze tracking systems, the cursor is positioned on a video screen according to the calculated gaze of the computer operator. A number of different techniques have been developed to select and activate a target object in these systems. In one example, the system activates a target object when it detects the operator fixating at the target object for a certain time. Another way to activate a target object is when the operator's eye blinks.
One problem with these systems is that humans use their eyes naturally as perceptive, not manipulative, body parts. Eye movement is often outside conscious thought, and it can be stressful to carefully guide eye movement as required to accurately use these target object activation systems. For many operators, controlling blinking or staring can be difficult, and may lead to inadvertent and erroneous target object selection. Thus, although eye gaze is theoretically faster than any other body part, the need to use unnatural activation (e.g., by blinking or staring) limits the speed advantage of gaze controlled pointing over manual pointing.
Another limitation of the foregoing systems is the difficulty in making accurate and reliable eye tracking systems. Because of eye jitter and other inherent difficulties in precisely monitoring eye gaze, only relatively large target objects can be activated by gaze-controlled pointing techniques. One approach to solving these problems is to use the current position of the gaze to set an initial display position for the cursor (reference is made, for example, to U.S. Pat. No. 6,204,828).
The cursor is set to this initial position just as the operator starts to move the pointing device. The effect for this operation is that the mouse pointer instantly appears where the operator is looking when the operator begins to move the mouse. Since the operator needs to look at the target object before pointing at it, this method effectively reduces the cursor movement distance. However, activation of the object requires the use of a pointing device such as a mouse, requiring the user to move a hand from a keyboard to the mouse or otherwise disturb the natural process of using the keyboard.
Another approach to activating graphical objects with eye gaze uses the eye-gaze as an input channel without a manual input device for the entire control process. Reference is made to Salvucci, D. D., and Anderson, J. R. (2000). Intelligent Gaze-Added Interfaces. Proc. CHI 2000: ACM Conference on Human Factors in Computing Systems, pages 273-280. Activation of the target object is performed using a special key such as the “enter” key. The cursor follows the user's gaze at all times. However, this moving cursor can be distracting to the user. Furthermore eye tracking precision can not be precise enough to locate some of the small GUI objects.
In particular, conventional eye gaze systems are limited as devices for activating objects on a graphical user interface (GUI). Gaze detection is not accurate enough to reliably activate target objects (or widgets) such as small buttons on a GUI. Further, eye gaze has both voluntary and involuntary aspects, making it difficult to use eye gaze as a pointing or activation device.
What is lacking is a technique that enables computer interaction with only gaze and keyboard. There have been some approaches to do this, but they rely on users intentionally treating gaze as a pointing device, which is unnatural and distracting in practice. What is therefore needed is to enable support for keyboard interaction concurrently with a fixated gaze on, or a natural look at the target, which happens to be what people do naturally. While interaction is typically thought of as selection and then activation, by interposing a navigation stage between these, the present invention enables the user to adjust for imprecision in the gaze tracking apparatus.